FIRST ORDER DEPENDENCE ON UNCERTAINTY SETS IN 
ROBUST OPTIMIZATION 



C.H. JEFFREY PANG 



Abstract. We show that a first order problem can approximate solutions of 
a robust optimization problem when the uncertainty set is scaled, and explore 
further properties of this first order problem. 
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1. Introduction 

Robust optimization is the methodology of handling optimization problems with 
uncertain data. In practice, the presence of uncertainties in optimization problems 
can make nominal solutions meaningless. Such uncertainties can come from data 
uncertainty in measurement and estimation, or from uncertainty in implementation. 
We refer to the recent text [3] for more details. 

Consider the linear program: 

mm x (Fx + d 
s.t. Ax < b. 

To account for the uncertainties in the data (A, b) , one instead considers a point x 
to be feasible if it satisfies 

Ax < b for all (A, b) E U. 

Here, U is a set containing the nominal data (A, b). We can consider the translation 
AU = hi — (A, b) and ask: What is the behavior of optimal solutions to the robust 
optimization problem if the set AU were to be scaled by some factor e? A large 
value of e corresponds to a more robust solution, and a small value of e places 
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more importance in the objective function. Understanding the dependence of e 
allows one to find a balance between optimization and robustness. The first order 
dependence on e is addressed in Corollary 12 . 41 for linear programs and Theorem 15. 31 
for nonlinear programs. 

The outline of this paper is as follows. We introduce robust linear programming 
in Section [2j Before we introduce robust nonlinear programming in Section |4l we 
recall some topics in variational analysis (or nonsmooth analysis) as presented in 
the texts [5J El H] in Section [3J In Section 21 we also define the tangential problem, 
which will be important in Theorem 15.31 our main result. We present first order 
properties of the tangential problem in Section [6j and study the effects of sums of 
uncertainty sets in the tangential problem in Section [7j 

2. Robust linear programming 

We keep our presentation compatible with [3J, and begin with the definition of 
the robust counterpart of a linear program. 

Definition 2.1. (Robust counterpart) For A € R mx ", b e R m ,_c € l n and rfel. 
where m > n, consider the linear program with parameters (A, b, c, d) 

(2.1) minc T a; + d 

X 

s.t. Ax < b. 

The robust counterpart (written as RC) of the above linear program is 

min < c(x) = sup [c T x + d] \ Ax < b for all (A, b, c,d) <EU 

x [ {A,b,c,d)£U 

= min{* | t > c T x + d, Ax < b for all (A, b, c, d) <E U}, 

x.t 

where ti is an uncertainty set for the parameters (A, b, c, d), with (A, b, c, d) £ IA. 

In a typical linear program, the variable d does not affect the minimizer, but 
one has to take perturbations in d into account in a robust optimization problem. 
The second formulation in the RC shows that we can rewrite the linear program so 
that c stays constant at c and d = 0. This is the approach we will take for the rest 
of this section, and we define U to be a set containing elements of the form (A, b) , 
where (A, b) are close enough to (A, b). For more details, we refer to [3J. 

We define AA, Ab and the set AU by the relations 

AA := A- A, 
Ab := b~b, 
and AU := U-(A,b). 

The vector x can be chosen so that it stays feasible under these first order pertur- 
bations. We write x — x + Ax. The RC is therefore simplified to 

min c T (x + Ax) + d 

Ax V ' 

(2.2) s.t. [Ai + AAi](x + Ax) < [k + Ab t ] for all (AA,, Ab t ) e AU U 
where AUi is the uncertainty in the ith row. 
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When AU is a small set , we seek to use a first order approximation to determine 
a robustly feasible x. Letting x = x + Ax, and removing the second order term 
(AAi) (Ax) in gives 

(2.3) Ai(x + Ax) + (AAi)x - (bi + Abi) < for all (AA h Abi) e AU U 
or \A % x - b] + A t (Ax) + [(AAi)x - Ab,] < for all (AAi, Abi) € AUi. 

If AiX — bi < and if AU were small enough, this constraint will not be tight in 
the optimization problem. With these in mind, we define the first order problem of 
a linear program. 

Definition 2.2. (First order problem) Let x be an optimal solution to p. 111 . The 

first order problem is the problem 

(2.4) minc T 7 

7 

s.t. A>7 + [(AAi)x - Ab] < 
for all (AAi, Abi) S AUi whenever AiX — bi = 0. 

The first order problem can also be written as 

T 

mine 7 

7 

s.t. An < - max \(AAi)x - AbA 

(AA,Ab)GAli 

whenever AiX — b L = 0. 

In the case where the optimal solution x is nondegenerate, i.e., when B = {i \ 
AiX — hi} is of size n and Ab is invertible, the optimal solution 7 of the first order 
problem is just 7 = A^w, where w is the vector 



- max( Aj4iAfc)eAW [(AA ll )a; - Ab 



nJ 



(2.5) 



•max(AA,A6)eAw[(A^4i„)x - Ab in ] 



where i±, . . .i n are the n elements in B. When x is a degenerate solution, the first 
order problem is still easy to solve. We illustrate with a particular example that 
the tangential constraints are easily obtained for rectangular uncertainty sets. 

Example 2.3. (Rectangular uncertainty) Suppose that the uncertainty set AU is 
rectangular, that is 

AU := {(AA, Ab) : \AA jik \ < \AbA < 5 3 

for all j e {1, . . . , m}, k E {1, . . . , n}}. 

Then for each i G B, 

71 

max{(AA,)x- Ab, \ (AAi, Abi) E AU t } = Si + ^2e tik \x k \. 

fe=i 



In Theorem 15 .31 we will discuss how an adapted first order problem gives a first 
order approximation of the solution to a robust optimization problem in a general 
setting of nonlinear programs. For now, we shall present the corollary in the simpler 
setting of linear programming. 



FIRST ORDER DEPENDENCE ON UNCERTAINTY SETS IN ROBUST OPTIMIZATION 4 



Corollary 2.4. (to Theorem 1 5. 3\) (First order approximation in linear program- 
ming) Consider the robust optimization problem 

minc T a: + d 

X 

(2.6) s.t. (A, + AA,)x < (h + Ab t ) for all (AA^Ab,) G eAU t for all i, 
and the first order problem 

min c T 7 

7 

(2.7) s.t. A»7 < - max UAAAx - Ab t ] 

for all i s.t. A^x = \>i. 

Let r be the set of optimal solutions to (|2.7p . Suppose 

(1) AUi are compact convex sets. 

(2) r is bounded. 

(3) There is some 7' such that A^' < whenever AiX — bi. 

(4) x is the unique minimizer of the nominal problem min{c T a; | Ax < b}. 

Then the set of cluster points of any sequence {-(x e — x)}, where x c is an optimal 
solution to (|2.6p and e — > 0, is a subset ofT. The objective value of (|2.6p , say v e , 
has an approximation v e = v + ev + o(e) ; where v is the objective value of (|2.7|) . 

In particular, ifT contains only one element, say 7, then lim e ^. (^e — ^) — 7> 
or x e € x + ej + o(e) . 

Proof. The condition that Q is Clarke regular at Ax — b holds in this case because 
is Clarke regular everywhere. The condition that x is the unique minimizer 
in (4) suffices because the domain is convex. The affine function x i— > c T x + d is 
locally Lipschitz and subdifferentially regular everywhere. □ 

3. Preliminaries in variational analysis 

In this section, we recall the definitions of some nonsmooth objects in variational 
analysis that will be necessary for the rest of the paper. We recall the definition of 
normal cones and Clarke regularity. 

Definition 3.1. (Normal cones and Clarke regularity) Let C C R™. For a point 

x G C, a vector v is normal to C at x in the regular sense, or a regular normal, 
written v G Nc{x), if 

v T (x — x) < o(\x — x\) for all x G C. 

It is normal to C in the general sense, or simply a normal vector, written v G Nc(x), 
if there are sequences Xi — > x and m — > v with m G Nc(xi). The set C is Clarke 
regular at x if Nq(x) = Nc(x). 

We refer the reader to [HJ Corollary 6.29] for equivalent definitions of Clarke 
regularity. The sets we will encounter in this paper are all Clarke regular, so this 
does not cause difficulties. 

We recall the definition of the tangent cone, which will be important in our main 
result. 
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Definition 3.2. (Tangent cones) The tangent cone of a set C C R m at some x G C 
is defined by 

Tc(x) := < w | — — — — > w for some Xi G C, U \ and xi — > 

Next, we recall sublinearity and equivalent definitions of subdifferential regularity 
that will also be useful for our main result. We take the definitions of subdifferential 
regularity from [9, Definition 7.25, Exercise 9.15, Corollary 8.19]. 

Definition 3.3. (positive homogeneity and sublinearity) A function h : R" — > R 
is positively homogeneous if h(Xx) = Xh(x) for all x and A > 0. It is sublinear if in 
addition 

h(x + x') < h(x) + h(x') for all x and x' . 
It is clear that sublinear functions are convex. 

Definition 3.4. (Subdifferential regularity) Let / : R™ -4 R be locally Lipschitz 
at x. 

(a) We say that the function / is (subdifferentially) regular at x if the epigraph 
epi/ :— {(x, t) | t > f(x)} is Clarke regular at (x, f(x)) as a subset of R" x R. 

(b) Define the subderivative df(x) : R™ — > R by 

(3.1) df(x)( W ) :=hminf /(g + TW) ~ /(g) . 

r\,0 T 

and the regular subderivative df(x) : R" — >■ R by 

;,,-w \ r f(x + Tw) - /(x) 

dj(xj(w) :— hmsup . 

t\o t 

x — >- X 

In general, the regular subderivative is sublinear. The function / is (subdiffer- 
entially) regular at x if and only if df(x) — df(x). Under subdifferential regu- 
larity, it is clear that the liminf in (|3.ip can be taken to be a full limit. Also, 
T cp Hf)(x,f(x)) =epi(df(x)). 

Since the tangent cone will play a major role in our main result, we now recall 
some calculus rules for tangent cones, highlighting a constraint qualification condi- 
tion similar to that of condition (j4|) in Theorem l5.3l The rest of this section will not 
be essential to the development of the paper, so one may skip to the next section 
in a first reading. We now recall a formula for tangent cones under intersections. 

Proposition 3.5. (Tangent cones to intersections) Let C — C\ l~l • • • PI C m for 

closed sets Ci C W 1 , and let x £ C. Suppose x is Clarke regular at Cj for all j. 
Assume either 

m 

(3.2) XjVj = 0, vj e N Cj (5) and Xj > for all j S {1, . . . , m} 

j'=i 

implies Xj = for all j G J', 
or equivalently: 

(a) there are no vectors {yj}" l = i such that yj _L Tc^x) and yx + • • • + y m = 
other than yj = for all j G {1, . . . , m}, and there is a vector w such that 
w G R™\{0} such that w G vint(T Cj (x)) for all j G { 1 , . . . , m}. 
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Then one has 

T c (x) =T Cl (x)n---nT Cm (S), 
and C is Clarke regular at x. 

Proof. Other than the equivalence of ()3.2p and (a), this result is stated in a more 
general case in [9j Theorem 6.42]. This result is obtained by consider the set 
D := d x • • • x C 2 C (R n ) m and the mapping F : x (->• (x, . . . , x) G (JR") m with 
X = R™ and applying (HI Theorems 6.31 and 6.41]. The constraint qualification 
condition required is (|3.2[) . By [9l Exercise 6.39(b)], (|3.2[) is equivalent to the 
existence of a u/ such that F(w') G rint(T£>(a;, . . . , x)) and having 

y _L Tr>{x, . . . , x) and F*(y) — implies y = 0. 

These conditions are equivalent to that in (a). □ 

We recall the Mangasarian-Fromovitz constraint qualification. 

Definition 3.6. (Mangasarian-Fromovitz constraint qualification) For C 1 functions 
f 3 ■ : R n -> M and j G {1, . . . , m}, let 

Q:={x£ K" | / 3 -(a:) < for all j £ {1, . . .,m}}. 

For x G C, let J' := {j \ fj(x) — 0}. The Mangasarian-Fromovitz constraint 
qualification (MFCQ) is satisfied at x if there is a vector w G M™ such that 

Vfj(x) T w < for all j G J'. 

Another equivalent definition of the MFCQ is the following "positive linear inde- 
pendence" condition 

XjVfj(x) = and Xj > for all j G J' implies Aj = for all j E J' . 

The classical definition of the MFCQ also takes into account equality constraints 
in the set Q, which we omit since they are not of immediate interest. 

To handle sets defined by nonsmooth constraints, we need to recall the subdif- 
ferential. 

Definition 3.7. (Subdifferentials) Consider a function / : K™ — > R such that / is 
locally Lipschitz at x. For a vector v G R™, one says that 

(a) u is a regular subgradient (also known as a Frechet subgradient) of / at x, 
written v G df(x), if 

/(a:) > /(*) + <w, a; - x) + o(\x - x\); 

(b) v is a (general) subgradient of / at x, written v G df{x), if there are sequences 
:Kj — > x and Ui — > u such that f(xi) — > f(x) and Vi G df(xi). 

(c) The set df(x) is the regular subdifferential, and the set df(x) is the (general) 
subdifferential. 

(d) The function / is (subdifferentially) regular at as if and only if df(x) = df(x). 

This characterization of subdifferentially regular functions is slightly different 
from the earlier definitions, but is equivalent in the case of locally Lipschitz functions 
in view of O Corollary 8.11, Theorem 9.13 and Theorem 8.6]. We shall only be 
concerned with subdifferentially regular functions throughout this paper, so there 
is no need to distinguish between df(x) and df(x). We conclude with results on 
the intersections of tangent cones described by constraints. 
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Proposition 3.8. (Tangent cone under constraints) Suppose C — {x fj(x) < 
0,j G J}, and J is a finite set. At the point x G C, let J' C J be the set of all j 's 
such that fj(x) = 0. If fj are continuous at x for all j G J, fj are continuously 
differentiable at x for all j G J' and the MFCQ is satisfied at x G C , then 

T c {x) = {z | Vf,{x) T z < for all j G J'}. 

In the nonsmooth case, if fj were locally Lipschitz and subdifferentially regular at 
x for all j G J' and 

(3.3) XjVj = 0, Vj G dfj(x) and Xj > for all j G J' 
je.J> 

implies Xj = for all j £ J , 
then x is Clarke regular at C , and 

(3.4) T c (x) = f]{z\v T z <0 for all v G df 3 {x)}. 

jeJ' 

= {z | v T z < for alive [j dfj{x)}. 

jeJ' 

Proof. We prove the general nonsmooth case for this theorem, which implies the 
smooth case. There is a neighborhood U of x such that C n U = [Hj^j'Cj] PI U, 
where Cj is defined by Cj — {x \ fj{x) < 0}. Furthermore, ^ dfj(x) for all 
j G J'. By [9l Theorem 10.3] (normal cones to level sets) and [H Corollary 6.29(d)] 
(tangent-normal relations in regular sets), Cj is Clarke regular at x, and the tangent 
cones (x) and normal cones Ncj (x) are given by 

N C] (x) = {Xv | A > 0, and v G dfj(x)}, 
and T Cj (x) = {w \ w T v < for all v G dfj(x)}. 
Therefore, condition (|3 . 3[) becomes 

Vj — 0, Vj G Ncj (x) implies Vj = for all j G J' . 

jeJ' 

By Proposition I3.5[ the tangent cone Tc(x) is 

T c (x) = f]T Cj (x), 
ie.J' 

which gives the formula for the tangent cone in the statement. □ 

It is well known that for the sets 

Ci := {(n,i 2 )eK 2 :J2>i?} 

C 2 := {(x u x 2 ) G R 2 : x 2 < -xj}, 

we have T Cl nc 2 (0) C T Cl (0) n T C2 (0) but the MFCQ is not satisfied. 

The constraint qualification condition (|3.3p can be checked by another equivalent 
condition when T"c. {x) have nonempty interior. 

Proposition 3.9. (Constraint qualification) Assume the conditions of Proposition 
\3.1A If TcAx) have nonempty interior and ^ dfj(x) for all j G J' , then the 
condition (|3.3p is equivalent to the existence of a vector w such thatw G int (Tc . (x)) 
(or equivalently w T v < for all v G dfj(x)) for all j G J' . 
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Proof. Recall that by [H Theorem 10.3], if ^ df(x), then the tangent cone Tc^x) 
is equal to {z \ z T v < for all v £ dfj(x)}, and the interior int (Tcr.(x)) is {z \ 
z T v < for all v £ dfj(x)}, which gives the equivalence on the conditions on 
w. Next, the equivalence of (|3.3j) and the condition in this result follow from 
Proposition 13.51 □ 

4. Robust nonlinear programming 
We look at nonlinear programs of the form 

(4.1) mm{c T x + d \ Ax -b £ Q}, 

X 

where Q C R fc is a closed set. Specifically, we consider problems of the form 

(4.2) min{c T a: + d \ A4X - h £ Qi, 1 < i < m}, 

X 

where Qi C M. ki are nonempty closed sets, A\ £ R fciXn , and bi £ R fci . We may 
write A as a concatenation of the matrices Ai and b as a concatenation of the 
vectors b,, and this would make (|4.1[) equivalent to (|4.2I) for Q = Qi X • • • X Q m and 

k = ki -\ h k m . One case of interest is the set Qi — {y \ fi,j{y) < for all j £ J} 

for some fij : M. ki — >■ K and the set J is finite. Another case of interest is conic 
programs, which arise when all Qi 's are closed convex pointed cones with nonempty 
interior. 

We now recall the definition of robust feasibility from [3]. 

Definition 4.1. (Robust feasibility) Let an uncertain problem be given and A1A = 
NU\ x • • • x AW m be a perturbation set. A candidate solution x £ W 1 is robustly 
feasible if it remains feasible for all realizations of the perturbation vector from the 
perturbation set, that is 

(4.3) [Ai + AAi]x - [k + Ab,} £ Qi V(i, 1 < i < m, (A^, A6<) £ Alii), 
where Alii C ( the uncertainty set in {Ai, bi). 

Remark 4.2. (Decomposing uncertainty sets) In the case where AU is not a direct 
product of uncertainty sets, the uncertainty sets AUi can be defined as 

AU t := {(AAi, Abi) : (AA^Abi) = EL.(AA,A6) for some (AA, Ab) £ AU}, 

where 11^ is the relevant projection from R kxn x R fe to R fci x " x R fci . It is clear that 
(|4.3p is equivalent to 

[A + AA]x — [6 + Ab] £ Q V(AA, Ab) £ AU. 

The definition for robust nonlinear programs encompasses nonlinear objective 
functions. 

Example 4.3. (Nonlinear objective) Consider the robust optimization problem 
min f(x) 

X 

s.t. [A\ + AAi]x - [bi + Abi] £ Qi V(i, 1 < i < m, (AAi, Abi) £ AUi). 
We can rewrite this robust problem as 
mini 

s.t. [Ai + AAi]x ~ [bi + Ab t ] £ QiV(i, 1 < i < m, (AA,, Ab,) £ AUi) 
and f(x) < t. 
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The function / is convex if and only if the epigraph epi(/) = {(x,t) | f(x) < t} 
is convex. Similarly, for a function / locally Lipschitz at x, the function / is 
sub-differentially regular at x if and only if epi(/) is Clarke regular at x. To prove 
our results for nonlinear functions, we can prove the result for linear objective 
functions and then appeal to the second formulation to obtain the result we need. 

The formula in the robust optimization constraint can be rewritten as 

[At + AAi](x + Ax) - [bi + Ab t ] e Q l 

[A lX - hi] + A t (Ax) + [(AAj)x - Ab t ] + (AA l )(Ax) e Q t . 

As in linear programming, we eliminate the second order term {AA){Ax) to obtain 
a first order approximation. For nonlinear programs, we also need to approximate 
the set Qi at AiX — bi by the tangential approximation Tq^Aix, — bi) + [AiX — bi] 
at A^ — bi. This gives our definition of the tangential problem. 

Definition 4.4. (Tangential problem) Let x be an optimal solution to a nonlinear 
programming problem with parameters (A, b) so that Q, is Clarke regular at AiX—bi 
for all i. The tangential problem to the robust optimization problem obtained with 
constraints as explained in Definition 14.11 is 

T 

nunc 7 

7 

s.t. [A\x - bi] + A a + [(AAi)x - Ab t ] e T Qi (A\x - 6<) + [A\x - k] 
for all (i, 1 < i < m, [AA U A6<) G ALU), 

or equivalently 

(4.4) minc T 7 

7 

s.t. Ai<y + [(AAi)x - Abi] G T Q% {A t x - k) 
for all (i, 1 < i < m, (AA t ,Ab z ) G AUi), 

which is also equivalent to 

T 

mine 7 

7 

s.t. A i7 + L t {AUi) G T Q% {A t x - bi) 
for all (i, 1 < i < m), 

where ^ : E m * xn xR mi -4 l m - is defined by L^AAi, Abi) = (AA^x-Ab,. We call 
the corresponding constraints to the tangential problem the tangential constraints. 

Remark 4.5. (Clarke regularity assumption) The assumption that each Qi is Clarke 
regular at AiX — bi in Definition 14.41 comes about because the set Q\ x • • ■ x Q m is 
Clarke regular at Ax — b if and only if Qi is Clarke regular at AiX — bi for all i, and 
in this case, 

T Qix - xQ m (Ax - b) = T Ql (Axx - l x ) X • • • x T Qm (A m x - b m ). 

(see [HI Proposition 6.41].) This property makes the tangential problem independent 
of how we decompose the set Q as a direct product of sets. 

We give some examples of tangential constraints. 
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Example 4.6. (Examples of tangential constraints) (a) When AiX — bi = and Qi 
is a closed convex cone, then Tq 1 (0) — Qi. In this case, the corresponding tangential 
constraint is obtained by just removing the second order term (AAi)(Ax). 

(b) When AiX — bi G int(Qj), then Tq^Aix — bi) = M. ki and the corresponding 
tangential constraint vanishes. 

In view of Example 14.61 we see that for linear programming, the tangential 
constraints and hrst order constraints are equivalent. When AiX — bi G dQi\{0}, we 
may still be able to calculate the tangential constraints using the material recalled 
in Section [3l 

We illustrate the tangential problem with the example on second order cone 
programming (SOCP). 

Example 4.7. (Tangential problem in SOCP) Consider the SOCP problem 

T 

mine x 

X 

s.t. AiX — bi G Qki for all 1 < i < m, 

where Qd C K d is the second order cone 

Qd ■= {w = (w Q , . . .,w d -i) G M. d | ||(«;i,. . . , Wd-i)h -w Q < 0}. 

Given an optimal solution x, we show how to obtain the tangential constraint. If 
AiX—bi G int(<5fej, then Tq^. (AiX — bi) = ~R ki by Example l4.61 and so the tangential 
constraint vanishes. If AiX, — bi = 0, then TQ k (AiX — bi) — Qk { by Example 14. 6[ so 
the tangential constraint is 

Aa + Li(AUi) c Q ki . 

We now consider the case A- L x — b L G 9Qfe i \{0}. Let z — AiX — b. L . In this 
case, zq = || (zi, . . . , Zfc ; _i) ||2- The gradient of the map (wo,w±, . . . ^w^-i) 
||(ti;i,...,*u fct -i)||2 — too at z is (-1, iifc^iiJ - Let R : M fc ' -> R k > be the 
reflection map 

R{w ,w 1 , . . . , w ki -i) ■= (-iu ,u>i, . . . , W ki -\), 

i.e., R multiplies the 0th coordinate by —1. The gradient at z can also be written 
as j^Rz. Therefore, by Proposition 13.81 

T Qki (AiX - bi) = {w G M fe * I w T Rz < 0}. 
Therefore, the tangential constraint is 

A n + [{AAi)x - Abi] G T Qh _ {AiX - h) for all (AA it Ah) G AU t . 
This can be written equivalently as 

z T RAa + z T R[{AAi)x - Abi] < for all (AAi,Ah) G Aid, 
or z T RA.a<- max z T RUAAAx - Ab t ]. 

(AAi,Abi)£AUi 

5. Main result: Approximation using the tangential problem 

In Theorem 15.31 we prove that if the uncertainty set in a robust optimization 
problem is dilated or expanded, then the robust optimal solution can be predicted 
from the exact solution of the nonrobust problem and the tangential problem. 

We now prove a lemma needed for the proof of our main result. 
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Lemma 5.1. (Compact sets in convex cones) Let D C R™ be Clarke regular at 
and C C R" be a compact convex set such that int(TD(0)) 7^ and C C Td(0). Let 
v £ int(Tc(0)). Then for all sufficiently small S > 0, there exists e > such that 
C + Sv + S 2 M C \D for all e £ (0, e]. 

Proof. Since v £ int(T D (0)), v + SB C int(T.D(0)) for all sufficiently small S > 0, 
and therefore C + Sv + S 2 M C int(To(0)). For every point w <E C + 6v + <5 2 B, we 
can find a convex polyhedral set P w such that int(P„,) 7^ and P w C mt(Tn(0))- A 
compactness argument shows that the set C + Sv + S 2 M is contained in the interior 
of finitely many of these convex polyhedral sets, so there is a convex polyhedral set 
P such that C + Sv + S 2 M C P C int(T£,(Q)). 

By the recession properties of tangent cones and the Clarke regularity of D, 
there is an e > such that econv({0} UP) C fl for all e 6 [0, e] (see Exercise 
6.34(a)]. The roots of this result on local recession vectors can be traced back to 
[7J.). Therefore C + Sv + S 2 M C conv({0} U P) C \D for all e G [0, e] as needed. □ 

We also need material in set- valued analysis as presented in [HI Chapters 4 and 
5] for the proof of Theorem 15.31 

Definition 5.2. [SI Definition 5.4] (Set-valued continuity) We say that S is a set- 
valued map, denoted by S : R™ =$ M. m , if S(x) C R™. A set-valued map S is outer 
semicontinuous (osc) at x if 

limsupS , (x) C S(x), 

x^rx 

or equivalently limsup^^j S(x) — S(x), but inner semicontinuous (isc) at x if 

liminf Six") D S(x), 

x^x 

or equivalently when S is closed- valued, lim inf S(x) = S(x). It is called con- 
tinuous at x if both conditions hold, i.e., if S(x) — > S(x) as x — > x. Here, the outer 
limit limsup^^j S(x) and the inner limit liminf S(x) are defined by 

limsupS'(x) := {u \ 3a;, — > x,3ui — > u with Ui € S(xi)} 

x^x 

liminf S(x) :— {u \ Vxi x, 3{ui} with Ui € S(Xi) 

x^x 

s.t. u is the limit of a subsequence of {ui}}- 

If S maps to compact sets, continuity as defined by inner and outer limits above 
is equivalent to continuity in the Pompieu-Hausdorff distance, which is a metric in 
the subset of compact sets. We refer to [9] for more details. We also need to recall 
the definition of epi- convergence. A sequence of functions hi : M." — > R is said to 
epi-converge to a function h : R" — > R, written hi A- h, if epi(/ij) — > epi(h). The 
history of epi-convergence can be traced back to the 1960's, and the result we need 
for our proof ([HI Theorem 7.33]) can be traced back to Salinetti (unpublished, but 
reported in [S]) and [2]. See d Chapter 7]. 

Here is our theorem on the approximation properties of the tangential problem. 

Theorem 5.3. (Approximation properties of Tangential problem) Consider the 
robust optimization problem 

min f(x) 

X 

(5.1) s.t. [Ai + AAi]x - [bi + Abi] e Qi for all (AAi, Afy) € tAUi for all i, 
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and the tangential problem 

min df(x)(j) 

7 

(5.2) s.t. A tl + [(AAj)x - A6<] G T Qi (A t x - k) 

for all (AAi,Abi) £ AU { for all i. 

Let f be the set of optimal solutions to (|5.2p . $ = {x \ Ax — b £ Q}, and x be a 
solution of the nominal problem min{/(a;) | Ax — b £ Q}. Suppose 

(1) Qi are closed sets that are Clarke regular at AiX — bi, 

(2) AUi are compact convex sets 

(3) r is bounded. 

(4) There is some 7' such that A,^' 6 int^Tq^ (A^x — bi)) for all i. 

(5) (Compactness) If {xi} C $ and f{xi) — > f(x), then Xi —¥ x. 

(6) / is locally Lipschitz and subdifferentially regular at x. 

Then the set of cluster points of any sequence {j(x € — x)}, where x e is an optimal 
solution to (|5.ip and e \ 0, is a subset ofT. The objective value of (|5.1j) , say v t , 
has an approximation v e = v + ev + o(e) ; where v is the objective value of (|5.2p . 

In particular, iff contains only one element, say 7, then Mm^o - (x c — x) = 7, 
or x e £ x + ej + o(e) . 

Proof. The proof of this result is broken up into four steps. In steps 1 to 3, we 
prove this result for the affine function f(x) = c T x + d, and df(x)(j) — c T x. In 
step 4, we use the observation in Example 14.31 to treat the case where / is locally 
Lipschitz and subdifferentially regular at x. 

Step 1: Rewriting the robust optimization problem (|5.1| . 

We rewrite the constraint in the robust optimization problem. 

[A,. + AA,]x - [bi + Abi] e Q t 
[Ai + AAi}x - AiX - Abi e Qi - [Mx - bi] 
<=> Ai(x -x) + [(AAi)x - Abi] + (AAi)(x - x) £ Qi - [A\x - bi]. 

Hence, 



AiX - bi £ Qi for all (AA i( A6») £ eAUi 
& Ai(x -x) + [{AAi)x - Abi] + {AAi)(x - x) £ Qi 
for all (AAi, Abi) € eAUi. 



[Ai 



The next step is to scale the variables AAi and Abi so that the e vanishes from the 
expression eAUi. This gives 



Ai(x -x) + [(AAi)x - Abi] + (A^j)(x - x) £ Qi 
for all (AAi, Abi) G eAiAi 



[Ax - h] 



<^ Ai 



[ ^(x-x) + [(AAi)x - Ah] + e(AAi) -(x-x) 
G - [Qi - [A lX - bi]] for all (AAi, Ab z ) £ AU t 



(5.3) <^ Aa e + [(AAi)x - Abi] + e(AAi)j e 



G - [Qi - [A lX - k]] for all (AAi, A6,) G AU U 
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where r y e := -(x — x) in the final expression. We see that as e \ 0, the expressions 
in (|5.3p converge to the corresponding expressions for the tangential constraints. 

Let r e denote the set of all feasible 7 e for the robust problem with parameter 
e, and T denote the set of all feasible 7 for the tangential problem. Similarly, let 
r e and r denote the set of optimal solutions to the corresponding problems. It is 
elementary to check that the sets r e , T, r € and T are all closed. 

Step 2: lim eN0 T £ = T. 

Suppose that {jj } is a sequence of feasible solutions to the robust problem with 
parameter tj , that is jj G T tj . Then each 7^ satisfies the formula in (|5.3[) with 
parameter tj . It is clear that any limit of {7^ } is a feasible solution of the tangential 
problem (|5.2p . so limsup e ^ o r £ C T. 

Next, we show that liminfe^o T c D T. Suppose 7 G T. We need to show that for 
any choice of tj \ 0, we can find 7, G T e . such that 7 = lim^oo 7^. Recall that 7 
satisfies 

where the linear map Li : W" HXn x R m > -)■ R TO< by Li(AAi, Abi) = (AAj)x - 
Abi. It follows from the convexity of AUi that Li(AUi) is convex. Since A^' G 
hit (Tq ; (JiiX — bi)), we can apply Lemma liOl to tell us that for all sufficiently small 
S > 0, there is some e > such that [A l (l J r5i)+L l (AU i )+8 2 W\ C \[Qi-(AiX-bi)] 
for all e G [0, e]. Therefore 

Ai(fr + */) + Li(AUi) + S 2 M C-[Qi- [Aix-bi]]. 

If e < e and emax{||AAf || | (A A;, Abi) 6 AUi}\\{j + Sy)\\ < S 2 , then for all 
(AAj, A&j) g ALU, 

(5.4) A( 7 + 5i) + [(AAi)x - Ab t ] + e(AA i )(7 + h') 

G + Si) + LiiAUi) + ellAA.ll ||(7 + Sj')\\B 

C Ai{j + Sj') + Li(AUi) + S 2 M 

C - [Qt - [AiX - bi]] . 

With this observation, we can choose a sequence Sj \ such that (7 + SjY) G T ej , 
which gives liminfex^o r e D T as needed. 
Step 3: limsup^Qfe C f. 

Recall that for a closed set D C R™, the indicator function 8d ■ R™ ->tU {00} 
is defined by 

'0 ifzGL* 
00 otherwise. 



S D (x) := 



Define h e : R n ->■ K and /i : M n -> R by /i e ( 7 ) := c T 7 + ^,(7) and 6,(7) := 
c T 7 + <5r(7). Since T e -> T, we have epi(<5 r J = T e x [0, 00) — > T x [0, 00) = epi(^r) 
(by O Exercise 4.29(a)]), so in other words Sr e A #r- By [9l Exercise 7.8(a)] we 
have /i e A- ft-. We seek to apply [H Theorem 7.33], which gives us the result we 
need. Before we can do so, we have to check that {h e } is eventually level bounded, 
that is, for any a G M, we have U[ 0je ]{7 | h € {i) < a} being bounded for some e > 0. 

Recall $ = {x | Ax — b G Q}. By Proposition 16. 6[ the boundedness of T 
is equivalent to {c}^ n T$(x) = {0}. Suppose on the contrary that {h e } is not 
eventually level bounded. Then there is some a and sequences {e^} and {7^} such 



FIRST ORDER DEPENDENCE ON UNCERTAINTY SETS IN ROBUST OPTIMIZATION 14 

that ti \ 0, {7i} is unbounded, and /i ei ( 7 ,) < a (or equivalently, tj G r £; and 
c T 7i < a). Let us write Xi — x + e^i- Since Xi G $ and 

(Fx < C T Xi — C T X + €iC T Ji < C T X + €iOt, 

we have c T Xi — > c T x. Note that this also gives us a > 0. By our compactness 
assumption, xt — > x, which means that \ei"fi\ — > 0. This means that t^t converges to 

a vector 700 in T$(0)\{0}. Observe that c T ^oo = lim^oo c T t% < li m i->oo pjry = 0, 

and that c T 7i > 0, so c T 7oo = 0. This is a contradiction to {c}^ D = {0}. 

We can thus apply O Theorem 7.33] to conclude that min{c T 7 | 7 G r c } converges 
to min{c T 7 | 7 G T} and limsup^oFe C T, ending the proof of the theorem for 
the linear case. 

Step 4: Locally Lipschitz subdifferentially regular / at x. 

Consider the problem 

min t 

(5.5) s.t. [Ai + AAi]x - [bi + Ah] G Q t for all (AA^Abi) G tAUi for all i, 
and (x,t) G epi(/) = {(x,t) \ f(x) < t}. 

and the tangential problem 
min s 

(5.6) s.t. A a + [(AAi)x - Abi] G T Qi {A z x - h) 

for all (AAi,Abi) G AUi for all i. 
and ( 7 ,s) G T cpi(f) (xJ(x)). 

The robust and tangential problems are equivalent to the respective problems 
(|5.ip and (|5.2p in the statement of the theorem. We now show that if conditions 
(1) to (5) in the theorem statement are satisfied for (|5.1[) and (|5.2p . then these 
conditions hold for (|5.5[) and (|5.6[) as well. 

For condition (1), we need only to check that epi(/) is Clarke regular at (x, f(x)), 
which is immediate from the subdifferential regularity of / at x. Condition (2) is 
straightforward. For condition (3), we note that the set of minimizers of (I5.6P is 
just r x {df(x)(j)}, where 7 is any element in 7. The set T x {d/(a;)(7)} is bounded 
if and only if T is bounded. 

We further assume that / is locally Lipschitz at x with Lipschitz modulus k. For 
condition (4), suppose 7' is a vector such that j4j 7 ' G int(TQ i (Aix — bi)) for all i. 
Since (0,1) G mt(T e p 1(/) (S, f(x))), we have (7', (k + 1)| 7 '|) G int(T opi(/) (x, f(x))), 
which verifies condition (4). 

We also need to check that given f(xi) —> f{x) and {xi} C $ implies Xi — > x, 
we have the compactness condition that ti — > f(x), (xi,U) satisfies f{xi) < U and 
{x^ C <& implies (xi,U) — »■ (x,f(x)). Since f(x) < f{xi) < U and ti — > /(a:), we 
have f(xi) — > /(x), which gives — > a;, and thus (a;^, tj) — > (a;, f(x)) as needed. □ 

We now take a closer look at step 4 of the proof of Theorem 15.31 Consider the 
general case where T e = argmin{c T 7 | 7 G r e }, T = argmin{c T 7 | 7 G T} and 
r = linigN^ore. It may turn out that limsup cX ^ o r e C T, as the example in Figure 
15.11 shows. Example 17.51 shows that it is possible for F to be bounded but not be a 
singleton set. In such cases, it is possible that r e is a singleton set, which occurs 
when the function / is strictly convex for example. 
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Figure 5.1. lim e ^ r e = r, but lim sup e ^. T e c r. 

We make an observation on the condition A^' G int(TQ i (AiX — bA) for all i in 
Theorem [ 



Example 5.4. (Constraint qualification in tangential problem) The optimization 
problem 

min X2 

s.t. x G Qi :— {x | — X\ + x\ < 0} 
x G Q-2 '■— {x | x\ + x\ < 0} 

can be written equivalently as 

min X2 

s.t. x G Qi nQ 2 = {(0,0)}, 

and the solution for both problems is x = (0,0). The tangential approximation for 
the first problem is 

min 72 

7 eR 2 



7 G {0} x 



s.t. 7 G T Ql (x) =M + x B 
7 G Tq 2 (x) =I_xI 

while the tangential approximation for the second problem is 



mm 72 

7 GM 2 

s.t. 7eT QinQ2 (x) = {(0,0)}. 

Clearly, the solutions for the two problems are different. This example shows that 
depending on how the optimization problem is written, the tangential problems 
may not be equivalent and may have different solutions. But note that in this case, 
^QinQ 2 (x) C Tq ± (x) n Tq 2 (x), which implies that the MFCQ does not hold, which 
in turn implies that there is no vector 7' such that A^' G mt(TQ i (AiX — bi)) for all 
i. The condition A^' G int(TQ ; (AiX — bi)) for all i in Theorem 15 .31 ensures that the 
MFCQ holds whenever a group of the {{Ai, bi)}™ =1 contains repetitions, ensuring 
that the tangent cones of the intersections is the intersections of the corresponding 
tangent cones through Proposition 13.51 

Theorem 15.31 shows that the decrease in objective function of the robust opti- 
mization problem is differentiable in the size of the uncertainty set at e = 0. This 
observation can help give an approximate of the maximum robustness one can af- 
ford if the objective is to be above a certain value. The tangential problem also 
shows that the variables that we should estimate or measure more accurately are 
those which make the set L(AU) = {(AA)x - Ab \ (AA, Ab) G AU} small. For 
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example, if x — 0, then more effort should be spent on determining b accurately 
rather than entries in A. Likewise, x determines which variables in A should be 
measured more accurately than others. 

Besides these analytical properties, Theorem 15.31 shows that solving the tangen- 
tial problem can help to obtain a good approximate of the robust solution. The 
robust optimization problem is known to be more computationally expensive than 
the original problem, so it will take more effort to obtain a desired level of accu- 
racy. With the tangential problem, we can make use of the previously calculated 
optimization problem to obtain an approximate of the robust solution. Such an 
approximation is likely to be simpler than the robust optimization problem (for 
example, for LP in (|2.5|) and for SOCP in Example 14. 7\ though it still may be com- 
putationally difficult), and need not be computed to very high accuracy to obtain 
a good approximate of the robust optimization problem. 

Remark 5.5. (Relaxing the constraint qualification in Theorem 15.31) The existence 
of 7' such that Ay' E hit (Tq (Ax — b)) can be relaxed slightly if more structure is 
known about the set Q. All we need is for the chain of inclusions (|5 .4[) to hold. For 
example, if Qi is polyhedral and there is some vector 7' such that 

Aii 6 rint(span(n i (AW i ))) n T Qi {A iX - b t ), 

where Hi(AUi) = {AAi | (AAj,A6j) s AUi for some Abi}, then the chain of in- 
clusions (|5.4[) would hold as well. A finer analysis on II,; (AM) and local recession 
vectors can give stronger results. 

6. First order optimality conditions of the tangential problem 

In this section, we discuss first order optimality conditions of the tangential 
problem, which can be useful for designing specialized numerical methods for the 
tangential problem. In view of Theorem 15.31 we also give sufficient conditions for 
f to be bounded and for f to be a singleton. 

As explained in [3j Chapters 5-8], the robust optimization problem is computa- 
tionally tractable if either Q is polyhedral or AU is polyhedral, while most other 
problems encountered in practice are not computationally tractable. Recall that a 
typical constraint in a robust optimization problem whose nominal solution is x is 

[A + AA](x + Ax) -[b + Ab}eQ for all (AA, Ab) e AU. 
If there were no uncertainty in the matrix A, then the constraint can be written as 
(6.1) A(Ax) - Ab e Q - [Ax - b] for all (AA, Ab) e AU. 

Recall that the tangential constraint is of the form 

A-y + [(AA)x - Ab] e T Q (Ax ~ b) for all (AA, Ab) € AU. 

If the variable Ax in (|6.1j) were replaced by 7, then we see that ()6.1|) is similar to 
the tangential constraint. In other words, the uncertainty in A is transferred to b in 
the tangential constraint through (AA)x. Tangential problems are still often hard 
to compute efficiently, but the additional structure may be exploited for designing 
specialized methods. The case where Tq (Ax — b) is polyhedral is just robust linear 
programming and is easy, while the tangential problem for L(AU) being polyhedral 
reduces to optimizing over the cone Tq(Ax — b), as illustrated below. 
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Example 6.1. (Polyhedral L(AU)) Suppose L(AU) is a polyhedral compact set 
of the form L(AU) — conv(t>i, . . . , vj). The tangential problem 

minft,(7) 

7 

s.t. Ay + L(AU) C Tq(Ax - b), 
where h is sublinear, is equivalent to 

min ft. (7) 

7 

s.t. A-y + vj £ Tq(Ax - b) for all 1 < j < J. 
For this section, we define the sets $ and \E' by 

(6.2) $ := {x:Ax-beQ}. 
vf := {y :y + L{AU) cT Q (Ax 

Hence T can be written similarly as 

(6.3) r = { 1 :A 1 + L{AU) C T Q (Ax 

= {7 : A-y £ 

Recall also that 

(6.4) f = argmin{d/(5)(7) 7 G T}. 

Proposition 6.2. (Tangent space of feasible set) Suppose Q is Clarke regular at 
Ax — b. If there is a vector 7' such that Ay' £ rint(Tg(Aa; — b)), then T<$(x) = {7 : 
Ay £ Tq(Ax — b)}, where $ is defined in (|6.2p . 

Proof. This follows directly from [9, Theorem 6.31]. The constraint qualification in 
that Theorem is equivalent to the condition in our statement through [9, Exercise 
6.39(b)]. □ 

The normal cone ^(7) can be estimated from the image Ny(A'y). 

Proposition 6.3. (Normal cone of tangent feasible set) Suppose there is a vector 
7' such that Aj' £ int(TQ(Ax — b)), and Q is Clarke regular at Ax — b. Then the 
normal cone N-p(j) equals {A T v | v £ -/V^(yl7)} ; where T and ^ are defined in 
dUg) and (jO]) . 

Proof. Note that T can be written in terms of ^> as T — {7 : A'y £ The result 
follows directly from [HI Theorem 6.14], though we still have to check the constraint 
qualification condition there. Through [SJ Exercise 6.39(b)], the constraint qualifi- 
cation condition required is that there is a vector 7' such that A'y 1 £ int(r$ (^7)). 
Note that the recession cone of ^> is Tq(Ax — b). This means that Tq(Ax — b)c 
Tqj(Aj), which shows that Ay' £ \yA,(Tq{Ax— b)) implies the constraint qualification 
condition. The conclusion is straightforward. □ 

One notices that if there is a vector 7' such that Ay' £ int (Tq(At — b)) and 
G mt(L(AU)) (which holds when G mt(AU)), then the tangential problem is 
feasible. 

If a vector a; is a solution of the problem min{/(2:) | x £ £>}, where / is locally 
Lipschitz and subdifferentially regular at x then it is well known that there is a 
c G df(x) such that — c G Nd(x) (see (9j Theorem 8.15] for example). We prove 
the following lemmas, whose proofs do not seem easy to find. 
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Lemma 6.4. (Strict minimizers) Let D C M" be Clarke regular at x, f : R n — > R 
be locally Lipschitz and subdifferentially regular at x. Then — c G int(A^£>(a;)) for 
some c G df(x) implies that x is a strict local minimizer o/min{/(x) | x G D}. 

Proof. Since — c G hxt(Nu(x)), there is some 5 > such that —c + 5M C Nd(x), 
and so there is a neighborhood O of x such that for x G I? n O, 



\x — x\ 



(x 



< 



c T [x — X s ) > 



By restricting O if necessary, for x E D P\0, we have /(x) > /(x) + df(x)(x — x) — 
f |x — x|. The well known characterization of the subderivative in terms of support 
functions (see for example [4, 6, 9J) gives 



f(x) > f{x)+df(x)(x-x)--\x-x\ 

= f(x)+ max v T (x — x) |x • 

vedf(x) '4' 

> f(x)+c T (x-x)--\x-x\ 

> f(x) + -\x-x\. 



Therefore x is a strict local minimizer. 



□ 



Lemma 6.5. (Equivalence of strict minimizer condition) Let D be Clarke regular 
at x. When f is C 1 at x, V/(x) ^ and — V/(x) G Njj(x) (which holds when x is 
a local minimizer o/min{/(x) | x G D}), then the conditions — V/(x) G int(iV£>(x)) 
and {V/(x)}^ PI Td(x) = {0} are equivalent. 

Proof. The fact that — V/(x) G -ZVd(x) when x is a local minimizer of min{/(x) | 
x G D} is well known (see Theorem 6.12] for example). 

Suppose that -V/(x) G mt(N D (x)), and v G {V/(x)} ± n T D {x). Then v G 
Trj(x) = [^^(x)]* (the polar cone of Nu(x)). In other words, v T s < for all 
s G Nd{x). If v ^ 0, since — V/(x) G int(iVD(x)), there is some e > such that 
V/(x) + ei> G Nrj(x), which gives v T (V/(x) + ev) < 0, and thus v T Vf(x) < 0. 
Since u G {V/(x)} x , this means v = 0, so {V/^)}- 1 n T D (x) = {0}. 

Next, suppose -V/(x) G 7V D (x)\int(7V D (x)). Then ^(x) (~V/(x)) D {0}. 
(See for example [9j Exercise 6.19].) Let w G AT JVd(s) (-V/(x))\{0}. Firstly, 
-AV/(x) G N D (x) for all A > 0, so w T (-AV/(x) - (-V/(x))) < for all A > 0, 
that is (1 - A)w T V/(x) < for all A > 0. This implies that w T Vf{x) = 0, or 
w G {V/(x)} ± . Secondly, w G N Nd ( S - ) (— V/(x)) means that w T (s + V/(x)) < 
for all s G Nrj(x). Since w T V f(x) — 0, this means that w T s < for all s G Nrj(x), 
which means that w G Td{x). Therefore, w G [{V/(x)} ± nTo(x)]\{0}, which gives 
us the equivalence between the two conditions. □ 

In Proposition 16.61 below, we show that the conditions in Lemma [6^41 can give us 
boundedness information on the robust problem. We also give conditions for which 
r is a singleton. 
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Proposition 6.6. (Conditions for T and optimality) Suppose that Q is Clarke 
regular at Ax — b, and there is a vector 7' such that Ay' G vat{TQ{Ax — &)). Let f 
be locally Lipschitz and subdifferentially regular at X, and x be a minimizer of the 
tangential problem (15. 2p . Recall also $, , T and T as defined in (|6.2p . (|6.3I) and 

(EH). 

(a) The set T is bounded if there is some c € df(x) such that — c G int(iV$(S)). 

(b) If f is C 1 at x and V/(x) ^ 0, then T is bounded if and only if — V/(x) G 
int(iV$(x)), which is also eguivalent to {Vf(x)} J ~ (~lT$(x) = {0}. 

(c) A feasible 7 £ T is in T if we can find c G df{x) such that — c G Nt^{x){i)- 
The condition — c G Ntq(x)('Y) holds when we can find {u{\ k i=l and {Wj}f = i 
such that Vi G ^Vr Q ( J fe_5)('7 + M « G L(AU) and — c = ^2i =1 XiA T Vi 
for some Aj > 0, 1 < i < fc. Here, L is the linear map L(AA, Ab) = 
{AA)x -Ab. 

(d) TTie set f is a singleton if we can find some c G df{x) such that — c G 
int(ATx' # (g)(7)) /or some 7 G T. T7ie condition — c G int^jVx^ (g)(7)) fto/rfs 
to/ien we can /md {m,}* =1 and {t>i}jLi such that Vi G AT Tq ^Ax—b) 0^7 + u »)> 
Ui G L{AU), k — dim(Q), {A T Vi}^ =1 is linearly independent, and — c = 
Si=i K-A T Vi, where Xi > /or all 1 < i < k. 

Proof. Part (a): Seeking a contradiction, suppose that T is unbounded, so there 
is a sequence {7^} of solutions of minimizcrs of (|5.2I) such that |7j| — >• 00, with 
-> 7oo- Note 

7i e r 

= {7: Aj + L(AU) cT Q (Ax-b)} 
C {7: A 7 eT Q (is-&)} 
= T$(x), 

so 7oo G T$(x). On the other hand, since d/(x)(-) is continuous and positively 
homogeneous, we have 

d/(x)( 7oo ) = hmd/(x)(^) 

= lim T ^-4f(x)(7 i ) 
= 0. 

The well known characterization of the subderivative in terms of support functions 
(see for example [U0IH]) gi ves 

max w T 7oo = df(x)('y oa )=Q 
vedf{x) 

=> c T 7oo < 0. 

But — c G int(iV$(x)), so c T joo > 0. This contradiction tells us that the set f is 
bounded as needed. 

Part (b): In view of part (a) and Lemma 16.51 we just need to prove that 
if f is bounded, then {V/(a;)} ± Pi T$(x) = {0}. We prove the contrapositive. 
Suppose to G [{V/(5)} x n T$(x)]\{0}. Then Vf(x) T w = 0, and to G T$(x), that 
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is Aw G Tq(Ax — b) by Proposition 16.21 Let 7 be some element in T. Then 
A(Xw + 7) + L{AU) = AXw + [A*i + L{AU)] 

C T Q (Ax-b) + T Q (Ax-b) 
= Tq(Ax - b). 

Therefore 7 + Ait; G T for all A > 0, which shows that T is unbounded, concluding 
our proof. 

Parts (c), (d): Note that 

f = {7 : 7 + u G T Q {Ax - b) for all u G L(AU)} 
C {7 : 7 + Ui G Tq(A5 - 6) for 1 < i < £;} 

= f| [T Q (AS - 6) -ui], 



so 



Ny(A-y) D N n k =i[TQ{M _i ) _ u . ] (A'y) 

3 [iV rQ(As _ 5) _„ 1 (l7)] + ' ' ' + W TQ {As-b)-uM^] 

(by [U Theorem 6.42]) 
= [N TQ (A S -bMl + mi)] + • • • + [N TQ(As _- b) (Aj + u fc )] 



3 |^A^i|Ai>o| 



Therefore, by Proposition! 

7V r ( 7 ) = {^ T y :j/ GiV*(A7)} 



For part (c), the condition stated is equivalent to the existence of c G df(x) such 
that — c G {X)j=i ^i v i I — 0}, which implies — c G iVr(7"), which in turn implies 
7 G f. Part (d) follows by applying Lemma ROl □ 

The conditions (c) and (d) in Proposition 16.61 can be helpful for designing nu- 
merical methods for solving the tangential problem. Due to Clarke regularity, the 
tangential problem is convex. However, the problem of determining whether a point 
is feasible is not necessarily easy. 

The result corresponding to conditions (c) and (d) in Proposition ^. 61 can also be 
generalized for robust optimization in general. The following result on normal cones 
in robust optimization combined with results on optimality of nonlinear programs 
(in Lemma 16.41 for example) gives us the optimality conditions. The proof of the 
following result is a direct application of O Theorems 6.14 and 6.42] similar to the 
proofs of Propositions 16.31 and 16. 6[ so we shall only state the result. 

Proposition 6.7. (Normal cones in robust optimization) For f : W l R, consider 
the robust optimization problem 

min f(x) 

X 

s.t. [A + AA]x - [b - Ab] G Q for all (AA, Ab) G AU. 
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Let the sets Q(AA, Ab) C R n and E C R™ be defined by 
Vt(AA, Ab) := {x \[A + AA]x - [B - Ab] G Q}, 

and 5 := {x \ [A + AA]x - [b - Ab] G Q for all (AA, Ab) G AU} 
p| Q(AA, Ab). 

(AA,A6)eAW 

Let i£S. 

(1) If Q is Clarke regular at [A + AA]i — [b — Ab] and the only vector y 6 
Nq([A + AA]x - [6 - Ab]) for which [A + AA] T y = is y = 0, then 

Nn(AA,Ab)(x) = {[A + AAfy \ y eN Q ([A + AA]x - [b - Ab])}. 

(2) For any finite set {(AAi, Abi)}i<=i C AU such that x, is Clarke regular 
at tt(AAi, Abi) for all i £ I, the normal cone N^(x) satisfies N~.{x) D 

7. Addition of uncertainty sets in the tangential problem 

For much of this section we focus on the tangential robust problem on addition 
of uncertainty sets. More specifically, we ask what we can say about the tangen- 
tial problem with uncertainty set AiA5i + A2A1S2 given knowledge of the optimal 
solutions of the tangential problem for the uncertainty sets A5i and A1S2 ■ Such a 
problem can arise from having to considering robust optimization problems with 
errors which are a sum of two or more unknown sources. 

We begin with some elementary properties. 

Proposition 7.1. (Elementary properties of uncertainty sets) Suppose Q is Clarke 
regular at Ax — b. For an uncertainty set AU, suppose the solution of the tangential 
problem is defined by 

v(AU) :— min/i(7) 

7 

s.t. + [{AA)x - Ab] e T Q (Ax - b) for all (AA, Ab) G AU, 
where h is sublinear. Then 

• If AS' = XAS, then v(AS') = Xv(AS). 

• If AS' D AS, then v(AS) < v(AS'). 

• // /i( 7 ) = c T 7 and AS' = AS + {s}, then v(AS') = v(AS) + c T s. 

We have the following result to study how set addition affects the solution to the 
tangential problem. 

Proposition 7.2. (Set addition) Recall the definition ofv(AU) in Provosition M .1\ 
Suppose AS = AiA5i + A2A1S2, where Ai,A2 > 0. Then v(AS) < Xiv(AS\) + 
\2v(AS 2 ). 

Proof. In view of Proposition 17. 11 we only need to prove the case for Ai = A2 = 1. 
Suppose 71 and 72 are solutions to 

min h (7) 

7 

s.t. Aj + [{AA)x - Ab] e T Q (Ax - b) for all {AA, Ab) e ASj 
for i = 1,2. If (AA, Ab) e ASi + AS 2 , then 

{(AA)x - Ab] = [(AAi)i - Ah] + [(AAi)x - Ab 2 ], 
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for some [(AAi)x — Abi} G ASi for i = 1,2. This means that 
A(^ + j 2 )-[(AA)x-Ab] 
= [Ay! - [(AAi)i - Ah}] + [Ay 2 - [(AA 2 )x - Ab 2 }] 
G T Q (Ax-b) + T Q (Ax~b) 
= Tq(Ax — b). 

So 71 + 72 is a feasible, though not necessarily optimal, solution to the tangential 
problem where the uncertainty set is ASi + AS 2 , which shows that v(AS) < h(y± + 
72) < h(yi) + h(y 2 ) = u(ASi) + v(AS 2 ). □ 

In a nondegenerate linear programming problem, we do have equality in Propo- 
sition 17.21 The assumption that A is an invertible square matrix below is not 
restrictive, because this is exactly what happens in a nondegenerate linear pro- 
gram. 

Proposition 7.3. (Set addition in nondegenerate linear programming) Consider 
the tangential problem having a linear programming structure 

v(AU) := minc T 7 

7 

s.t. Ay + [(AA)x - Ab] < for all (AA, Ab) G AU. 

Assume that A is square and invertible, and — c = X)i=i ^i-^i f or some A,; > 0, 
l<i<k. Then v(ASi + AS 2 ) = v(ASi) + v{AS 2 ). 

Proof. Recall that feasibility can be rewritten as 

Aiy + max UAAAx - AbA < for all i. 

(AA,Ab)£AU 

Form the vector w(AU) G W n by 

\w(AU)}i := - max KAAAx - AbA. 

(AA,Ah)GAW 

The value v{AU) is equal to c T A~ 1 w(AU). The condition on — c in the statement 
assures that this minimizer is unique through convexity and Lemma 16.41 It is clear 
that for each i, we have 

max HAAAx - AbA 

(AA,Ab)eASi + AS 2 

= max UAAAx - AbA + max HAAAx - AbA, 

(AA,Ab)eASi (AA,A6)6AS 2 

which implies that w(ASi + AS 2 ) — w(ASi) + w(AS 2 ). This immediately gives 
u(A5i + AS 2 ) = v(ASi) + v(AS 2 ) as needed. □ 



The next step is to ask whether the property in Proposition 17.31 is satisfied for 
problems of the form 

T 

mine 7 

7 

s.t. Ay + [(AA)x ~ Ab] G Q for all {AA, Ab) G AU, 

where Q is a convex cone. If Q = {y \ Ay < 0}, where A is an invertible square 
matrix, then the constraint above can be transformed into 

A[Ay + [{AA)x - Ab}] < for all (AA, Ab) G AU, 

and we can apply Proposition 17.31 on A A. 
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In the general case, the equality v(ASi + AS2) = v(ASi) + v(AS2) may not hold. 
We give two examples to illustrate this. In the first example, we have a degenerate 
linear program, while in the second example, we have different cones for a conic 
programming problem. 

Example 7.4. (Inequality in sums of uncertainty sets 1) Consider the problem 
v(AU) := min X2 

x£R 2 

s.t. (A + AA)x - (0 + Ab) < for all (AA, Ab) e AU, 

.where .1 = I 1 -1 I . h = 0. 

0.5 -1 

The optimal solution to the nonrobust problem is x — 0, and the tangential problem 
is 

v(AU) := min 72 

7 gb 2 

s.t. Aj + [(AA)0 - Ab] < for all (AA, Ab) e AU. 

We illustrate this example in Figure I7TT1 Let the uncertainty sets ASi and AS2 be 
defined by 

ASi := {0} x {Ab : Ah = 0, |A6 2 | < 4e, A6 3 = 0} 
AS 2 := {0} x {Ab : Abt = 0, Ab 2 = 0, |A6 3 | < 3e} 
A5i+A5 2 = {0} x {A6 : A61 = 0, |A6 2 | < 4e, |A6 3 | < 3e}. 

To find v(ASi), we note that tangential problem becomes 




The first two rows are the active constraints, which gives a solution of 7(A«Si) = 
(— 2e, 2e) and v(ASi) — 2e. To find v(A6>2), a similar set of calculations shows that 
the first and third constraints are the active constraints, which gives 7(AiS2) = 
(— 2e, 2e) and v(AS2) = 2e. Similarly, v(ASi + AS2) — 2e, and all constraints are 
active. We have v(AS± + AS2) < v(ASi) + v(AS2) as needed. 

Here is a second example for the case when the cone Q is slightly more compli- 
cated. 

Example 7.5. (Inequality in sums of uncertainty sets 2) Consider the problem 
v(AU) := min xr 

7 GR 3 

s.t.(J + AA)x - (0 + Ab) e Q for all (AA, Ab) e AU. 

Here, the matrix A — I is the 3x3 identity matrix, b £ M 3 is the zero vector, and 
the convex cone Q is defined by 

Q := {x e M 3 : £ 3 > max(|xi|, |x 2 |)}. 
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FIGURE 7.1. Illustration of Example 17.41 

The optimal solution to the nonrobust problem is x = 0, and the tangential problem 
is 

v(AU) := min 73 

7 eR 3 

s.t.7 + [(AA)0 - Ab] G T Q (0) for all (AA, Ab) G AU. 
Let a < b and the sets 1S1 and S 2 be defined by 

ASi := {0}x{A6eM 3 :|Afe 1 |<^,|A6 2 |<^|A6 3 |<^} 
AS 2 := {0}x{A6gM 3 :|A6 1 |<^,|A6 2 |<|,|A6 3 |<^} 

AS 1 +AS 2 = {0} x |A6e K 3 : |A&i| < ^t^,|Ab 2 | < ^y^,|A6 3 | < <j}. 

See Figure I7T21 for an illustration of the convex cone Q, and the projection of A<Si, 
A1S2 and ASi + AS2 onto the 2-dimensional space corresponding to the first 2 
coordinates in R 3 . It is elementary that v(ASi) = w(A5 2 ) = | + |, and v(ASi + 
AS 2 ) = ^ + 6. This gives u(A5i + A5 2 ) < u(A5 x ) + u(A5 2 ). If the calculations 
had been performed with the second order cone Q 1 defined by 

Q' := |x G M 3 : .x 3 > \jx\ + x%} 
instead, then we also get the conclusion v{AS\ + AS 2 ) < v(ASi) + v(AS 2 ). 
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FIGURE 7.2. These diagrams illustrate Example 17.51 The figure 
on the left illustrates the convex cone Q in R 3 , while the figure on 
the right illustrates the first 2 coordinates for the shapes AS± and 
A5 2 . 
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